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Foreword 


This  report  describes  an  approach  to  measuring  total  organizational  effectiveness  that  focuses 
on  the  value  of  what  an  organization  does.  The  approach  integrates  measures  of  productivity, 
quality,  and  other  less  tangible  dimensions  of  performance  into  a  formula  that  produces  an  index 
of  “total  value.”  Once  this  total  value  is  determined,  it  can  be  used  as  the  basis  for  a  gain  sharing 
payout,  or  as  an  index  of  improvements  in  total  quality. 

This  report  is  one  of  a  series  of  reports  on  measurement  issues  related  to  productivity  gain 
sharing  and  total  quality.  The  other  reports  in  this  series  are:  (a)  Using  Performance  Indexing  to 
Measure  Organizational  Gains  in  a  White  Collar  Environment  (Tatum  &  Nebeker,  1996), 
(b)  Examples  of  White  Collar  Measurement  Using  a  Typology  of  Organizational  Effectiveness 
(Nebeker,  Tatum,  &  Wolosin,  1996),  and  (c)  Integrating  Measurement  Approaches  in  Gain  Sharing 
and  Total  Quality  (Tatum,  Shaw,  &  Main,  1996). 

This  effort  was  conducted  under  the  support  of  the  Office  of  the  Assistant  Secretary  of  Defense, 
(Personnel  and  Readiness),  Workforce  Quality  and  Productivity  Division. 

Point  of  contact  regarding  this  effort  is  Dr.  B.  Charles  Tatum,  Personnel  and  Organizational 
Assessment  Department,  Code  12,  Navy  Personnel  Research  and  Development  Center,  San  Diego, 
CA  92152-7250,  (619)  553-7955,  DSN  553-7955. 
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Director 
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Summary 


Problem  and  Background 

The  Federal  Government  has  recognized  the  need  to  direct  efforts  to  establish,  improve,  and 
maintain  quality  and  productivity.  To  do  this,  organizations  must  devise  performance  measurement 
systems  that  accurately  reflect  quality,  productivity,  and  general  effectiveness.  Productivity  Gain 
Sharing  and  Total  Quality  Management  require  that  accurate  performance  indicators  be  used  to 
measure  productivity  and  quality  improvements. 

Objective 

The  purpose  of  this  report  is  to  provide  a  measurement  approach  that  integrates  a  wide  variety 
of  performance  measures.  The  approach  allows  the  user  to  generate  an  index  of  ‘  total  value  that 
can  be  used  to  assess  improvements  and  make  gain  sharing  payouts. 

Approach 

The  methodology  assumes  the  user  has  established  reliable  and  valid  performance  indicators 
(see  Nebeker,  Tatum,  &  Wolosin,  1996)  for  a  discussion  of  how  to  develop  performance 
indicators).  A  formula  is  presented  that  allows  the  user  to  enter  their  performance  indicators  into 
the  equation  and  create  an  index  of  “total  value.”  The  index  is  an  aggregation  of  many  measures 
across  a  wide  range  of  performance  dimensions  (e.g.,  productivity,  quality,  financial  performance). 
The  index  of  “total  value”  is  an  expression  of  the  total  performance  of  an  organization. 

Conclusions 

The  effectiveness  of  an  organization  is  more  than  just  the  quantity  and  quality  of  its  products 
and  services.  Organizations  must  develop  their  resources,  keep  their  stakeholders  happy,  and  invest 
in  their  future  if  they  expect  to  survive  in  today’s  economy.  Consequently,  organizations  must 
attempt  to  measure  intangible  dimensions  (e.g.  customer  satisfaction  or  employee  morale)  as  well 
as  their  outputs  if  they  are  to  receive  an  accurate  picture  of  performance.  The  approach  presented 
in  this  report  is  one  method  for  assessing  organizational  performance  that  includes  both  tangible 
and  intangible  dimensions. 
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Introduction 


How  an  organization  performs  is  the  product  of  many  different  components.  One  approach  for 
classifying  these  diverse  components  of  organizational  performance  is  outlined  in  two  reports 
issued  by  the  Navy  Personnel  Research  and  Development  Center  under  the  sponsorship  of  the 
Office  of  the  Assistant  Secretary  of  Defense  (Nebeker,  Tatum,  &  Wolosin,  1996;  Tatum,  Nebeker, 
&  Wolosin,  1996).  In  these  reports,  organizational  performance  is  divided  into  four  basic 
components — ^productivity,  financial  performance,  stakeholder  relations,  and  resource 
development.  Each  of  these  four  basic  components  are  then  subdivided  into  smaller  units  of 
performance  (Tatum,  Nebeker,  &  Wolosin,  1996),  and  examples  of  specific  measures  of  the 
performance  components  are  presented  (Nebeker,  Tatum,  &  Wolosin,  1996).  By  examining  these 
two  reports,  an  organization  gets  a  very  good  idea  of  how  to  measure  various  aspects  of  their 
performance,  and  what  is  involved  in  trying  to  improve  their  performance  and  become  a  more 
effective  organization.  The  next  step  in  becoming  a  more  effective  organization  is  to  develop  a 
systematic  approach  for  combining  a  set  of  performance  measures  and  arriving  at  a  number  (or 
index)  that  summarizes  how  the  organization  is  performing  as  a  whole.  This  summary  number  can 
then  be  monitored  over  time  and  the  organization  can  chart  improvements  and  target  areas  in  need 
of  improvement.  A  report  by  Tatum  and  Nebeker  (1996)  illustrates  how  this  index  can  be  computed 
using  a  technique  known  as  “performance  indexing.”  This  report  offers  an  alternative  approach. 

The  measurement  approach  proposed  here  integrates  measures  of  the  quality  and  quantity  of 
production  (what  is  called  Output  Value  [OV])  with  measures  of  the  more  intangible  qualities  of 
an  organization  (e.g.,  resource  development,  stakeholder  relation)  that  relate  to  its  long  term 
success  and  survival  (these  measures  are  referred  to  collectively  as  System  Value  [SV]).  With  this 
approach,  the  productivity  of  an  organization  is  measured  by  the  familiar  ratio  of  inputs  to  outputs. 
However,  the  value  (or  quality)  of  the  outputs  and  the  value  of  the  system  as  a  whole  is  made  an 
explicit  part  of  the  measurement  system. 

Steps  to  Measuring  Total  Organizational  Value 


Product  Value 

To  begin  with,  the  organization  must  know  its  products  and  services.  For  each  product  or 
service,  there  exist  attributes  that  define  the  value  or  quality  of  the  product  or  service.  To  use  a 
simple  example,  if  the  product  were  fruit,  the  quality  attributes  might  be  color,  texture,  smell,  taste 
(etc.)  To  use  a  more  complex  example,  if  the  product  were  a  computer  program  used  in  a  missile 
guidance  system,  the  quality  attributes  might  be  the  results  of  various  tests  of  accuracy, 
functionality,  failure  rate,  ease  of  use  (etc.).  Quality  attributes  are  inherent  in  services  as  well  as 
products.  If  an  organization  delivers  a  service  such  as  training  or  repairs,  for  example,  the  quality 
attributes  might  be  course  evaluation  results,  customer  satisfaction,  or  on-time  delivery. 

When  the  organization  knows  its  products  and  services,  and  can  measure  the  quality  attributes 
in  a  valid  and  reliable  way  (see  Nebeker,  Tatum  &  Wolosin,  1996)  for  a  discussion  of  validity  and 
reliability  of  measurement),  then  they  are  in  a  position  to  calculate  the  Product  Value  (PV)  (we 
will  refer  to  this  component  as  PV,  but  keep  in  mind  that  what  we  are  describing  applies  to  services 
as  well  as  products).  PV  is  the  weighted  sum  of  all  the  quality  attributes  for  a  single  product  and  is 
defined  mathematically  as: 
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E(i) 

i  =  1  =  aibi  +  a2b2  +  •  •  •  +  anb^ 

Where, 

PV  =  Product  Value 

n  =  The  number  of  attributes 

a,  =  Standardized  measure  of  the  i  th  product  attribute  (where  i  can  take  on  values 
1  through  n) 

bj  =  Weighting  factor  for  attribute  at  the  i  th  level  (where  i  can  take  on  values 
1  through  n) 

a^  =  The  first  attribute  for  the  product 
a2  =  The  second  attribute  for  the  product 
ajj  =  The  last  attribute  for  the  product 
bi  =  The  weight  of  the  first  attribute 
bj  =  The  weight  of  the  second  attribute 
bn  =  The  weight  of  the  last  attribute 

The  above  formula  simply  means  that  you  must  take  all  of  the  attributes  for  a  given  product, 
weight  them  by  importance,  and  add  them  together.  The  attributes  must  be  standardized  in  some 
fashion  so  that  they  will  all  be  scaled  to  the  same  metric.  There  are  several  techniques  for 
standardizing  measures,  but  the  most  common  is  the  Z  score  which  can  be  found  in  any  elementary 
statistics  text  The  weighting  factor  (b)  is  used  because  the  quality  attributes  are  not  equally 
important  to  the  value  of  the  product.  For  example,  if  failure  rate  is  the  most  important  quality 
dimension,  and  other  dimensions  such  as  ease  of  use  or  self  life  are  less  important,  then  the 
weighting  factor  for  failure  rate  should  be  larger  than  the  other  two.  There  are  many  ways  to  derive 
these  weightings  (e.g.,  the  cost  of  the  attribute,  principle  component  analysis,  conjoint  analysis, 
expert  judgment)  but  it  is  beyond  the  scope  of  this  report  to  outline  these.  Nevertheless,  one  must 
be  aware  of  the  need  for  deriving  weightings  if  not  all  attributes  are  equally  important.  Of  course, 
if  all  the  attributes  are  of  equal  importance,  then  the  b  factors  can  be  excluded  and  PV  is  merely 
the  sum  of  all  the  standardized  measures  for  the  attributes. 

Value  of  Production 

The  Value  of  Production  (VP)  refers  to  the  sum  total  of  all  products  and  services  of  a  given 
type  created  by  an  organization  in  a  certain  period  of  time.  For  example,  if  an  organization  delivers 
software  products,  the  VP  would  be  the  total  number  of  software  packages  produced  in  any  given 
period  of  time.  In  other  words,  the  VP  is  the  sum  of  the  Product  Values  (PVs).  Mathematically, 
the  VP  is  defined  as: 


VP  =  XpV  or  m  =»=  PV 
j  =  1 


(2) 


2 


Where, 

VP  =  Value  of  Production 
m  =  The  number  of  products  of  a  given  kind 
PV  =  Product  Value  (computed  in  equation  [1]  above) 

PV  =  The  mean  (average)  of  all  Product  Values 

Output  Value 

The  VP  captures  the  quantity  and  quality  of  production  for  a  particular  product  or  service.  The 
Output  Value  (OV)  is  the  weighted  sum  of  different  product  (service)  lines.  For  example,  if  an 
organization  not  only  produced  software  packages,  but  also  delivered  training  and  repaired 
computers,  these  different  product/service  lines  would  need  to  be  combined  to  capture  the  value  of 
all  the  outputs  of  the  organization.  Combining  these  different  lines  is  accomplished  by  computing 
a  weighted  sum  of  the  separate  VP.  Mathematically,  the  OV  is: 

h  ») 

OV  =  X  VPfcBfc  =  VP,Bi  +  VP2B2  +  •  •  ■  +  VP^Bh 

k  =  1 


Where, 


OV 

h 

VPt 

Bk 

VP, 

VP2 

VPh 

B, 

Bj 

Bh 


Output  Value 

The  number  of  different  product  lines 

Value  of  Production  of  the  k  th  product  line  (where  k  can  take  on  values 
1  through  h) 

Weighting  factor  for  VP  of  the  k  th  product  line  (where  k  can  take  on  values 
1  through  h) 

Value  of  Production  of  the  first  product  line 
Value  of  Production  of  the  second  product  line 
Value  of  Production  of  the  last  product  line 
The  weight  of  the  first  product  line 
The  weight  of  the  second  product  line 
The  weight  of  the  last  product  line 


The  above  formula  means  that  you  must  take  all  of  the  separate  product  lines,  weight  them,  and 
add  them  together.  The  weighting  factor  (B)  is  used  because  it  is  unlikely  that  the  different  product 
lines  are  equally  important  to  the  organization.  For  example,  if  the  software  product  is  a  much  more 
important  line  than  training  or  repair,  then  the  weighting  factor  for  the  software  line  should  be 
larger  than  the  other  two  product/service  lines.  It  is  beyond  the  scope  of  this  report  to  suggest  all 
of  the  ways  in  which  these  weighting  factors  can  be  derived.  Nevertheless,  it  is  important  to  note 
that,  unless  all  product/service  lines  are  of  equal  importance,  these  weighting  factors  are  required. 
One  possible  technique  for  deriving  the  weights  is  to  determine  the  proportion  of  the  budget  that 
is  accounted  for  by  each  product/service  line. 
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System  Value 


The  effectiveness  of  an  organization  is  more  than  just  the  quantity  and  quality  of  its  products 
and  services  (its  OV).  Organizations  must  develop  their  resources,  keep  their  stakeholders  happy, 
and  invest  in  their  future  if  they  expect  to  survive  and  thrive  in  today’s  world.  Consequently, 
organizations  must  make  some  attempt  to  include  intangible,  long-term  dimensions  into  their 
performance  calculations.  For  example,  organizational  reputation,  absenteeism,  turnover,  or 
employee  morale  have  a  long  term  or  intangible  link  to  the  “bottom  line,”  but  these  “soft” 
indicators  of  effectiveness  must  be  factored  into  the  equation.  SV  is  the  sum  total  of  the  important 
intangibles  that  make  up  the  total  performance  of  an  organization.  An  organization  must  decide 
which  of  these  many  dimensions  to  include  in  the  formula,  and  what  weights  should  be  given  to 
these  dimensions.  In  large  part  these  decisions  are  governed  by  the  nature  of  the  organization.  If 
the  organization  is  a  service  organization,  then  overall  customer  satisfaction  and  employee  morale 
may  be  critical  to  the  total  performance  of  the  organization.  On  the  other  hand,  if  the  organization 
is  involved  in  a  heavy  industrial  activity,  investments  in  facilities  and  equipment,  or  employee 
health  and  safety  may  be  the  primary  concerns.  In  any  event,  calculating  the  SV  component  follows 
the  same  pattern  as  earlier  calculations  (i.e.,  it  is  the  weighted  sum  of  the  important  system 
dimensions).  Mathematically,  SV  is: 

P  (4) 

SV  =  dqWq  Ci][W2  4"  d2W2  4" ...  4"  ^p^p 

q  1 


Where, 

S  V  =  System  Value 

p  =  The  number  of  system  dimensions  (non-product  indicators  of  effectiveness) 

dq  =  Measure  of  system  value  of  the  q  th  dimension  (where  q  can  take  on  values 
1  through  p) 

Wq  =  Weighting  factor  for  the  q  th  dimension  (where  q  can  take  on  values  1  through  p) 
dj  =  The  first  system  dimension 
d2  =  The  second  system  dimension 
dp  =  The  last  system  dimension 
W 1  =  The  weight  of  the  first  system  dimension 
W2  =  The  weight  of  the  second  system  dimension 
Wp  =  The  weight  of  the  last  system  dimension 

As  in  previous  calculations,  the  weighting  factors  must  be  derived  in  such  a  way  as  to  reflect 
the  relative  importance  of  each  system  dimension.  Because  these  dimensions,  by  their  nature,  have 
an  uncertain  relationship  to  the  outputs  of  the  organization,  deriving  the  weighting  factors  will  be 
uncertain  as  well.  Over  time,  empirical  relationships  between  the  system  dimensions  and  various 
measures  of  output  and  financial  performance  can  be  established.  These  empirical  relationships  can 
then  be  used  to  establish  the  weighting  factors.  In  the  short  term,  however,  the  organization  must 
rely  on  the  expert  judgment  of  knowledgeable  people  within  the  organization  to  obtain  estimates 
of  the  weightings. 
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Total  Value 


For  any  given  performance  period,  the  total  value  (TV)  of  the  performance  of  an  organization 
is  a  combination  of  its  OV  and  its  S  V.  In  other  words,  the  value  of  all  the  outputs  plus  the  sum  of 
all  the  intangibles  constitutes  the  best  index  of  how  effectively  the  organization  performed  its 
mission.  The  final  calculation  is  simply  the  weighted  sum  of  the  OV  and  the  SV,  and  is  expressed 
in  the  following  mathematical  formula: 

(5) 

VT  =  OV  Wo  +  SV  Ws 

Where, 

TV  =  Total  Value 

OV  =  Output  Value 

Wq  =  Weighting  for  Output  Value 

SV  =  System  Value 

Wj  =  Weighting  for  System  Value 

The  weighting  factors  (W„  and  W^)  reflect  the  relative  importance  of  OV  and  SV  and  must  be 
derived  empirically  or  through  expert  judgment.  If  weighting  factors  cannot  be  determined,  one 
(usually  not  acceptable)  alternative  is  to  assume  that  OV  and  SV  are  equally  important  and  drop 
the  weighting  factors  from  the  equation. 


Using  Total  Value  to  Calculate  a 
Productivity  Gain  Sharing  Payout 

The  TV  can  be  calculated  for  any  given  performance  period  and  used  as  the  basis  for  a 
productivity  gain  sharing  (PGS)  payout.  In  a  sense,  the  TV  is  an  organization’s  best  indicator  of  the 
quantity  and  quality  of  its  products  and  services  during  a  particular  performance  period.  This 
indicator  also  takes  into  account  certain  dimensions  of  performance  that  reflect  its  long  term  health 
and  investment  in  the  future  (the  S  V).  To  use  TV  as  the  basis  for  PGS  payouts,  an  organization  must 
calculate  the  TV  over  several  performance  periods  and  establish  a  baseline  (generally,  baselines 
represent  several  quarters  of  performance).  They  must  also  track  the  cost  of  their  inputs  during  the 
same  period.  Once  a  baseline  of  outputs  (TV)  and  inputs  is  established,  the  ratio  of  these  outputs 
and  inputs  forms  the  baseline  productivity  required  for  PGS.  During  some  current  period, 
calculating  a  payout  is  simply  a  matter  of  calculating  the  TV  for  the  current  period,  determining 
the  costs  of  inputs  for  that  same  period,  and  then  plugging  these  numbers,  along  with  the  baseline 
figures,  into  a  PGS  formula  (see  Tatum  ,  Nebeker,  &  De  Young,  1996)  for  a  detailed  discussion  of 
how  to  use  a  PGS  formula). 


Conclusions 

This  approach  to  measuring  the  total  performance  of  an  organization  is  similar  to  other 
approaches  of  assessing  organizational  effectiveness.  Felix  &  Riggs  (1983)  have  described  what 
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they  call  the  Objectives  Matrix,  and  Pritchard  and  his  colleagues  (Pritchard,  Jones,  Roth,  Stuebing, 
&  Ekeberg,  1989;  Pritchard  &  Roth,  1991)  have  developed  a  methodology  called  Productivity 
Measurement  Enhancement  System  (ProMES).  Both  of  these  approaches  share  with  the  present 
approach  an  attempt  to  combine  a  set  of  diverse  measures  into  a  single,  composite  index  that 
reflects  the  total  performance  of  the  organization.  Each  method,  however,  has  a  different  emphasis. 
The  Objectives  Matrix  emphasizes  setting  goals  and  builds  its  approach  around  making 
improvements  relative  to  these  goals.  ProMES  also  has  a  goal  setting  process,  but  also  emphasizes 
contingencies  between  performance  indicators  and  overall  effectiveness,  as  well  as  feedback  to 
organizational  members. 

The  present  approach  emphasizes  integrating  the  less  typical  “system”  measures  (e.g., 
reputation,  morale,  absenteeism)  with  the  more  traditional  “production”  measures.  The  emphasis 
of  the  present  approach  is  unique  in  the  literature  on  measuring  organizational  effectiveness,  and 
adds  a  critical  dimension  to  assessing  total  organizational  performance.  The  information  contained 
in  this  document,  along  with  the  two  reports  on  the  typology  of  organizational  effectiveness 
(Nebeker,  Tatum,  &  Wolosin,  1996;  Tatum,  Nebeker,  &  Wolosin,  1996)  should  provide  an 
excellent  foundation  for  any  organization  attempting  to  measure  its  performance.  If  an  organization 
is  seriously  considering  PGS,  these  reports  on  measurement  should  be  required  reading. 
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